Joint Camera Pose Estimation and 3D Human Pose Estimation in a Multi-camera Setup

In this paper we propose an approach to jointly perform camera pose estimation and human pose estimation from videos recorded by a set of cameras separated by wide baselines. Multi-camera pose estimation is very challenging in case of wide baselines or in general when patch-based feature correspondences are difficult to establish across images. For this reason, we propose to exploit the motion of an articulated structure in the scene, such as a human, to relate these cameras. More precisely, we first run a part-based human pose estimation for each camera and each frame independently. Correctly detected joints are then used to compute an initial estimate of the epipolar geometry between pairs of cameras. In a combined optimization over all the recorded sequences, the multi-camera configuration and the 3D motion of the kinematic structure in the scene are inferred. The optimization accounts for time continuity, part-based detection scores, optical flow, and body part visibility. Our approach was evaluated on 4 publicly available datasets, evaluating the accuracy of the camera poses and the human poses.